The subject matter disclosed herein relates to a system to establish the trustworthiness of an autonomous agent and, more particularly, to a system to establish the trustworthiness of an autonomous agent by applying human patterns of trust building.
A significant barrier to more extensive use of autonomous agents is not the capability set, the features, the cost or the level of technological maturity of the autonomous agents, but rather the lack of trustworthiness of the autonomous agents. This trustworthiness problem at least partially stems from the fact that establishing the trustworthiness of autonomous agents has not generally been attempted and, even if attempts have been made, they have not been based on methods of establishing trust among humans. That is, current approaches fail to recognize trust and fail to use formal models of the trust and trust building relationships.
For example, over the past decade, the use of RPVs has become an essential part of US military strategy. In operation, RPVs are typically given “sliding autonomy” to operate with limited operator control where an autonomous agent controlling the RPV is allowed to function unless or until the operator feels he must take control. In practice, however, this scheme devolves to simple tele-operation since the operator(s) have no way to determine whether the RPVs can successfully operate autonomously and, thus, cannot trust the relevant autonomous agent to operate properly. In another technique, exhaustive tests of autonomous agents are performed during development stages. Such tests can identify and correct bugs in the autonomous agent but are useless in an operational environment where unanticipated stimuli are encountered and it is discovered that the autonomous agent cannot respond properly. A third approach is to deploy the autonomous agent and attempt to discern trustworthiness in particular situations by collecting trouble reports and making patches, if possible, on the fly.
In 2001, Congress approved a military doctrinal statement that set an aggressive goal for autonomous agents. That statement set forth that, by 2015, one-third of combat vehicles are to be autonomous. As of 2011, the reality has been that despite increasing UAV/RPV use, only limited autonomy (e.g. flight/navigation) is actually deployed.